Integrating Physical Qualities

The ubiquitous embedding of digital technologies in everyday environments and activities increases the complexity, variety and occurrence of human interactions with digital media. Furthermore, the advances in display technologies bring digital information and interaction possibilities on the very artifacts of our physical and social environments, such as tables and walls. For input and navigation into the digital space we obviously need physical handles in the analogue one, be they tangible (e.g., a mouse), or not (e.g., speech). In this sense we can consider every kind of interaction with digital media as hybrid in nature, since it encounters a physical as well as a digital component. Mouse and keyboard have mostly embodied those handles so far, enabling the manipulation of Graphical User Interfaces (GUIs) in the WIMP (Windows, Icons, Menu, Pointer) paradigm. But such a paradigm is inappropriate to cope with the heterogeneous, distributed, mobile, and multi-users scenarios of ubiquitous computing, and poorly reflects the richness of our physical interactions. Put differently, when digital information is displayed for a different interaction style (e.g., gesture-based interaction) and is embedded in different domains and physical artifacts (e.g., interactive tabletops, wall displays), novel handles, affordances, and paradigms need to be designed for users to ‘get a grip’, and ‘grasp’ the conceptual model in the diverse contexts of hybrid interaction. In my work I explore the affordances of physical interactions as resources for the design of hybrid ones. My goal is to inform the design of interactive systems so as to draw on such affordances, and finally to reduce the cost of users’ transition from novice to experts. The adoption and skilled use of a tool inevitably imply costs, e.g, the costs in terms of time for learning and experience. Similarly to the idea of technology transfer, if we are able to transfer a skill from one domain to another one, we can potentially reduce costs and bring innovation in the target domain. In other words, if we, as designers, are able to design tools that reflect existing human skills, we have a good chance to reduce the cost of acquisition of skill in the target domain. This is in line with Buxton’s metaphor of the three mirrors as human-centered criteria for assessing design [2]. According to this thesis, the more a design reflects humans’ physical, cognitive and social capabilities, the better the design supports such a skill transfer. Coming from a design background, my most natural approach is to tackle the problem from the physical angle, i.e., from the pragmatic level of the input interface (as discussed in [1]). This deals with gestures, spatial, as well as device issues, and is the first level of contact between the user and the system: in other words, it is our handle for hybrid interaction with the digital space. Such a level has an impact on the whole experience of interaction and it is in the scope of my research to leverage an understanding of the properties (and the implications thereof) of interactive systems at this level. To this end, in my work I am seeking to understand what it is about physicality, in terms of multisensorial as well as cognitive and emotional aspects, that affects the quality of hybrid experiences. In order to address these questions I focus on: • Identifying the affordances (comprehending cognitive, functional, sensorial and social affordances) of both physical and digital media in a systematic way; • Given that affordances are goal and context-dependent, understanding how the domain, the physical context and the social context can affect the perception of such affordances. • Understanding how the integration of physical affordances in a specific hybrid context is beneficial for the design of augmented, meaningful experiences, that go beyond the ones that are possible in the purely physical realm.